{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: Tongyi Qwen\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "# ChatTongyi\n",
    "Tongyi Qwen is a large language model developed by Alibaba's Damo Academy. It is capable of understanding user intent through natural language understanding and semantic analysis, based on user input in natural language. It provides services and assistance to users in different domains and tasks. By providing clear and detailed instructions, you can obtain results that better align with your expectations.\n",
    "In this notebook, we will introduce how to use langchain with [Tongyi](https://www.aliyun.com/product/dashscope) mainly in `Chat` corresponding\n",
    " to the package `langchain/chat_models` in langchain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "# Install the package\n",
    "%pip install --upgrade --quiet  dashscope"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Get a new token: https://help.aliyun.com/document_detail/611472.html?spm=a2c4g.2399481.0.0\n",
    "from getpass import getpass\n",
    "\n",
    "DASHSCOPE_API_KEY = getpass()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"DASHSCOPE_API_KEY\"] = DASHSCOPE_API_KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chat resp: content='Hello' id='run-f2301962-6d46-423c-8afa-1e667bd11e2b'\n",
      "chat resp: content='!' id='run-f2301962-6d46-423c-8afa-1e667bd11e2b'\n",
      "chat resp: content=' How' id='run-f2301962-6d46-423c-8afa-1e667bd11e2b'\n",
      "chat resp: content=' can I assist you today' id='run-f2301962-6d46-423c-8afa-1e667bd11e2b'\n",
      "chat resp: content='?' id='run-f2301962-6d46-423c-8afa-1e667bd11e2b'\n",
      "chat resp: content='' response_metadata={'finish_reason': 'stop', 'request_id': '921db2c5-4d53-9a89-8e87-e4ad6a671237', 'token_usage': {'input_tokens': 20, 'output_tokens': 9, 'total_tokens': 29}} id='run-f2301962-6d46-423c-8afa-1e667bd11e2b'\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.chat_models.tongyi import ChatTongyi\n",
    "from langchain_core.messages import HumanMessage\n",
    "\n",
    "chatLLM = ChatTongyi(\n",
    "    streaming=True,\n",
    ")\n",
    "res = chatLLM.stream([HumanMessage(content=\"hi\")], streaming=True)\n",
    "for r in res:\n",
    "    print(\"chat resp:\", r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/cheese/PARA/Projects/langchain-contribution/langchain/libs/core/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `BaseChatModel.__call__` was deprecated in langchain-core 0.1.7 and will be removed in 0.2.0. Use invoke instead.\n",
      "  warn_deprecated(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"J'adore programmer.\", response_metadata={'model_name': 'qwen-turbo', 'finish_reason': 'stop', 'request_id': 'ae725086-0ffa-9728-8c72-b204c7bc7eeb', 'token_usage': {'input_tokens': 36, 'output_tokens': 6, 'total_tokens': 42}}, id='run-060cc103-ef5f-4c8a-af40-792ac7f40c26-0')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "\n",
    "messages = [\n",
    "    SystemMessage(\n",
    "        content=\"You are a helpful assistant that translates English to French.\"\n",
    "    ),\n",
    "    HumanMessage(\n",
    "        content=\"Translate this sentence from English to French. I love programming.\"\n",
    "    ),\n",
    "]\n",
    "chatLLM(messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tool Calling\n",
    "ChatTongyi supports tool calling API that lets you describe tools and their arguments, and have the model return a JSON object with a tool to invoke and the inputs to that tool."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use with `bind_tools`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "content='' additional_kwargs={'tool_calls': [{'function': {'name': 'multiply', 'arguments': '{\"first_int\": 5, \"second_int\": 42}'}, 'id': '', 'type': 'function'}]} response_metadata={'model_name': 'qwen-turbo', 'finish_reason': 'tool_calls', 'request_id': '4acf0e36-44af-987a-a0c0-8b5c5eaa1a8b', 'token_usage': {'input_tokens': 200, 'output_tokens': 25, 'total_tokens': 225}} id='run-0ecd0f09-1d20-4e55-a4f3-f14d1f710ae7-0' tool_calls=[{'name': 'multiply', 'args': {'first_int': 5, 'second_int': 42}, 'id': ''}]\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.chat_models.tongyi import ChatTongyi\n",
    "from langchain_core.tools import tool\n",
    "\n",
    "\n",
    "@tool\n",
    "def multiply(first_int: int, second_int: int) -> int:\n",
    "    \"\"\"Multiply two integers together.\"\"\"\n",
    "    return first_int * second_int\n",
    "\n",
    "\n",
    "llm = ChatTongyi(model=\"qwen-turbo\")\n",
    "\n",
    "llm_with_tools = llm.bind_tools([multiply])\n",
    "\n",
    "msg = llm_with_tools.invoke(\"What's 5 times forty two\")\n",
    "\n",
    "print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Construct args manually"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='', additional_kwargs={'tool_calls': [{'function': {'name': 'get_current_weather', 'arguments': '{\"location\": \"San Francisco\"}'}, 'id': '', 'type': 'function'}]}, response_metadata={'model_name': 'qwen-turbo', 'finish_reason': 'tool_calls', 'request_id': '87ef33d2-5c6b-9457-91e2-39faad7120eb', 'token_usage': {'input_tokens': 229, 'output_tokens': 19, 'total_tokens': 248}}, id='run-7939ba7f-e3f7-46f8-980b-30499b52723c-0', tool_calls=[{'name': 'get_current_weather', 'args': {'location': 'San Francisco'}, 'id': ''}])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.chat_models.tongyi import ChatTongyi\n",
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "\n",
    "tools = [\n",
    "    {\n",
    "        \"type\": \"function\",\n",
    "        \"function\": {\n",
    "            \"name\": \"get_current_time\",\n",
    "            \"description\": \"当你想知道现在的时间时非常有用。\",\n",
    "            \"parameters\": {},\n",
    "        },\n",
    "    },\n",
    "    {\n",
    "        \"type\": \"function\",\n",
    "        \"function\": {\n",
    "            \"name\": \"get_current_weather\",\n",
    "            \"description\": \"当你想查询指定城市的天气时非常有用。\",\n",
    "            \"parameters\": {\n",
    "                \"type\": \"object\",\n",
    "                \"properties\": {\n",
    "                    \"location\": {\n",
    "                        \"type\": \"string\",\n",
    "                        \"description\": \"城市或县区，比如北京市、杭州市、余杭区等。\",\n",
    "                    }\n",
    "                },\n",
    "            },\n",
    "            \"required\": [\"location\"],\n",
    "        },\n",
    "    },\n",
    "]\n",
    "\n",
    "messages = [\n",
    "    SystemMessage(content=\"You are a helpful assistant.\"),\n",
    "    HumanMessage(content=\"What is the weather like in San Francisco?\"),\n",
    "]\n",
    "chatLLM = ChatTongyi()\n",
    "llm_kwargs = {\"tools\": tools, \"result_format\": \"message\"}\n",
    "ai_message = chatLLM.bind(**llm_kwargs).invoke(messages)\n",
    "ai_message"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tongyi With Vision\n",
    "Qwen-VL(qwen-vl-plus/qwen-vl-max) are models that can process images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=[{'text': 'The image presents a flowchart of an artificial intelligence system. The system is divided into two main components: short-term memory and long-term memory, which are connected to the \"Memory\" box.\\n\\nFrom the \"Memory\" box, there are three branches leading to different functionalities:\\n\\n1. \"Tools\" - This branch represents various tools that the AI system can utilize, including \"Calendar()\", \"Calculator()\", \"CodeInterpreter()\", \"Search()\" and others not explicitly listed.\\n\\n2. \"Action\" - This branch represents the action taken by the AI system based on its processing of information. It\\'s connected to both the \"Tools\" and the \"Agent\" boxes.\\n\\n3. \"Planning\" - This branch represents the planning process of the AI system, which involves reflection, self-critics, chain of thoughts, subgoal decomposition, and other processes not shown.\\n\\nThe central component of the system is the \"Agent\" box, which seems to orchestrate the flow of information between the different components. The \"Agent\" interacts with the \"Tools\" and \"Memory\" boxes, suggesting it plays a crucial role in the AI\\'s decision-making process. \\n\\nOverall, the image depicts a complex and interconnected artificial intelligence system, where different components work together to process information, make decisions, and take actions.'}], response_metadata={'model_name': 'qwen-vl-max', 'finish_reason': 'stop', 'request_id': '6a2b9e90-7c3b-960d-8a10-6a0cf9991ae5', 'token_usage': {'input_tokens': 1262, 'output_tokens': 260, 'image_tokens': 1232}}, id='run-fd030661-c734-4580-b977-b77d42680742-0')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.chat_models import ChatTongyi\n",
    "from langchain_core.messages import HumanMessage\n",
    "\n",
    "chatLLM = ChatTongyi(model_name=\"qwen-vl-max\")\n",
    "image_message = {\n",
    "    \"image\": \"https://lilianweng.github.io/posts/2023-06-23-agent/agent-overview.png\",\n",
    "}\n",
    "text_message = {\n",
    "    \"text\": \"summarize this picture\",\n",
    "}\n",
    "message = HumanMessage(content=[text_message, image_message])\n",
    "chatLLM.invoke([message])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
